REMARKS 

Claims 1-30 are pending in the application. Claims 1-30 stand rejected 
under 35 U.S.C. 103(a) as being unpatentable over Murakami et al. (US 
6,108,674), in view of Lieberman, "Autonomous Interface Agents", March 1997, 
and in fiirther view of Katz et al. (US 5,404,295). 

Applicant respectfully requests reconsideration in view of the foregoing 
amendments and the remarks hereinbelow. 
Rejection of Claim 1, 2. 4-6 under 35 U.S.C. 103(a): 

The Question of Obviousness of Combining Murakami et al. and Lieberman: 

In Murakami et al., a keyboard is used to enter the contents of a document. 
The user may enter a specific command to start an image output program, 
whereupon illustrations matching the contents of the document are inserted into 
the document. The Examiner notes correctly that Murakami et al. does not teach 
a system that continuously monitors the document contents to automatically 
propose images to the user for inclusion into the document, but the Examiner 
suggests (1) that this is taught by Lieberman and (2) that it would have been 
obvious to combine the teachings of Murakami et al. and Lieberman. Applicants 
respectfully traverse this conclusion. 

Lieberman discloses two types of agents: (1) interface agents that are 
programs that can affect the objects in a direct manipulation interface without 
explicit instruction from the user and (2) autonomous agents that operate in 
parallel with the user. Examples given by Lieberman on interface agents include 
tutoring systems and context-sensitive help systems. Lieberman's examples of 
autonomous agents include web browsing according to a user's prior interests. 
Lieberman does not suggest that agents can be used to retrieve stored images. It 
is the non-obvious recognition of this ability that is an important feature of the 
present invention. 

In Graham v. John Deere Co., 383 U.S. 1, 17, 148 USPQ 459, 467, 15 L. 
Ed. 2d 545, 86 S. Ct. 684 (1966), the Supreme Court requires that the Examiner 
must explain why the proposed modification would be obvious. Referring to the 
Office Action, the Examiner's reason for finding obviousness is "because it 
would supply the autonomous agent a continual supply of desired output image 
data based on selected keywords of textual files. " In other words, it would be 



obvious because it would be useful. Being "useful" is one of the requirements for 
patentability, not a sign of obviousness. 

The Courts require the Examiner to identify where the prior art provides a 
motivating suggestion for the modification, as for example, in the decision in In 
re Jones, 958 F.2d 347, 21 USPQ.2d 1941 (Fed. Cir. 1992), where the Court held: 

"Before the PTO may combine the disclosures of 
two or more prior art references in order to establish prima 
facie obviousness, there must be some suggestion for doing 
so ... In re Fine, 837 F.2d 1071, 1074, 5 USPQ.2d 1596, 
1598-99 (Fed. Cir. 1988). "[at 1943] 

"The prior art must provide one of ordinary skill in 
the art the motivation to make the proposed molecular 
modifications needed to arrive at the claimed compound, 
"[at 1944] 

Moreover, the Courts have advocated that even if the prior art may be 
modified as suggested by the Examiner, the modification is not obvious unless the 
prior art suggests the desirability for the modification, as for example, in the 
decision in In re Fritch, 922 F.2d 1260, 23 USPQ.2d 1780 (Fed. Cir. 1992), 
where the Court held: 

"Mere fact that prior art may be modified to reflect 
features of claimed invention does not make modification, 
and hence claimed invention, obvious unless desirability of 
such modification is suggested by prior art . . . "[at 1 780] 

"The mere fact that the prior art may be modified in 
the manner suggested by the Examiner does not make the 
modification obvious unless the prior art suggested the 
desirability of the modification. In re Gordon, 733 F.2d at 
902, 221 USPQ at 1 127. "[at 1783] 
It has been held that the motivating suggestion must be explicit, as was 
decided in the seminal case of Winner International Royalty Corp. v. Wang, No. 
96-2107, 48 USPQ.2d 1 139 (D.C.D.C. 1998), where the Court held: 

"... invention cannot be found obvious unless there 
was some explicit teaching or suggestion in art to motivate 



one of ordinary skill to combine elements so as to create 
same invention. n [at 1 140] 

"... there must have been some explicit teaching or 
suggestion in the art to motivate one of even ordinary skill 
to combine such elements so as to create the same 
invention, "[at 1 144] 
In the present application, the Examiner's mere agreement with the 
applicant that the invention provides a useful and desirable result cannot be 
considered to be evidence of obviousness. 

The Question of Reiving on Katz to Show Annotations: 

Claim 1 calls for the provision of suggested annotations from which the 
user selects for association with images. Referring to the specification, images 
could be annotated with text labels identifying the image subject matter, such as 
in a document, email note, or other human perceivable output. Katz, on the other 
hand, relates to computer retrieval of database material., and has nothing to do 
with annotation of images in output documents. Nor does Katz disclose the 
provision of suggested annotations to the user for the images. 

The Examiner's statement of what is disclosed by Katz does not reflect an 
understanding of what is claimed. The primary reference fails to disclose 
annotation of images in output documents by providing the user with suggested 
annotations for the images. Katz fails to disclose, in conceptual terms, the 
information undisclosed by the primary reference. Assuming arguendo that the 
references might be capable of combination, there is at least one limitation in the 
claimed invention that is not disclosed by the references individually or in 
combination. "Each element of a claim is material." Ashland Oil, Inc. v. Delta 
Resins & Refractories, Inc., 227 USPQ 657,666 (Fed. Cir., 1985) 

Rejection of Claims 3 and 8 under 35 U.S.C. 103(a): 

Claims 3 and 8 depends from Claim 1 and is patentable therewith. Claim 
3 further calls for automatically updating the metadata for matched images with 
suggested annotations. Claim 8 calls for the provision of the user with the 
capability of adding annotations to the matched images. Katz, which is cited by 
the Examiner as teaching annotations, does not suggest updating metadata with 
annotations. The Examiner points to a portion of Murakami et al. for this 



teaching, but, of course this is not possible since Murakami et al. does not relate 
to annotations. 

Rejection of Claim 7 under 35 U.S.C. 103(a): 

Claim 7 depends from Claim 1 and is patentable therewith. Claim 7 
further calls for determining the context based upon the proximity of typed words 
to the cursor location. The Examiner correctly states that Murakami et al. teaches 
insertion of an illustration at the cursor position; but fails to find a disclosure of 
determining the context based on the position of the cursor. Since not one of the 
references discloses the determination of the context based on the cursor location, 
there is at least one limitation in the claimed invention that is not disclosed by the 
references individually or in combination. 

Rejection of Claims 9-12 under 35 V.S.C. 103(a): 

Claim 9 calls for ranking the matched images according to context. The 
Examiner states that Murakami et al. teaches image retrieval based on three 
keywords, and that this disclosure "suggests" image retrieval ranking. Even if 
this "suggestion" were in fact true, it would not meet the terms of the claims, 
which require display of the images according to their rank. Claims 10-12 depend 
from Claim 9 and are patentable therewith. These dependent claims further 
define the invention by structure not found in the references of record. 

Rejection of Claims 13-30 under 35 U.S.C. 103(a): 

The Office Action does not offer independent grounds for rejection of 
Claims 13-30. Instead, the Office Action suggests that the rationale for their 
rejection is the same as for Claims 1-13 [sic.]. As such, Applicant can only refer 
the Examiner to the arguments for patentability set forth above with respect to 
Claims 1-12, and further point out that Claims 13-30 contain limitations not found 
in Claims 1-12 and not addressed by the Examiner. For example: 

• Claims 13-21 and 24 are limited to e-mail applications and the 
provision of the capability of moving matched images into the 
message window of e-mail, not disclosed in the prior art of record. 

• Claim 1 8 provides for ranking based on the number of matched 
keywords. 

• Claim 19 calls for ranking based on the temporal distance between the 
keyword and the associated metadata. 
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• Claim 20 recites basing ranking on a confidence level for manually 
selected keyword annotations. 

• Claim 2 1 requires ranking based on spatial or geographic 
correspondence between keyword and the metadata. 

Conclusion: 

It is respectfully submitted, therefore, that in view of the above remarks, 
that this application is now in condition for allowance, prompt notice of which is 
earnestly solicited. 
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